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Like so many moments of significant change within organizations, the Art Institute of 
Pittsburgh’s decision to implement rigorous program assessment was motivated by the promise 
of a carrot: Middle States accreditation. A little over a decade ago the college, which had been 
primarily recognized as an associate degree-granting career/technical school, decided to expand 
into bachelor’s degrees, creating a need to radically expand its General Education department 
and to examine course offerings and teaching practices within its major programs. In the spring 
of 2006, this work entered into a critical phase when AiP became a candidate for regional 
accreditation. It was quickly revealed that our assessment practices up to that point were 
heartfelt, but woefully inadequate. Therefore, it was decided that we needed a little help. 

The college hired the noted assessment experts James Nichols and Karen Nichols 
(fonnerly of the University of Mississippi) to act as consultants. They are the authors of General 
Education Assessment For Improvement of Student Academic Achievement: Guidance for 
Academic Departments and Committees and The Departmental Guide and Record Book for 
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Student Outcomes Assessment and Institutional Effectiveness, among others. While neither 
Nichols was a specialist in writing assessment, their wide range of successful consulting 
experiences were generally admired at AiP. As we began to plan our department’s approach to 
assessment, especially writing program assessment, my idea was to follow a model I’d been part 
of at LaRoche College where faculty shared student portfolios and used them as a point of 
discussion about problems in writing and the classroom and about pedagogical best practices. 
While this was not truly qualitative assessment, I felt this was an important and potentially viable 
alternative to quantitative, rubric-based assessment. However, when this model was proposed to 
the consultants, Jim Nichols ultimately rejected it. While Nichols could see the advantages of 
such an approach, it was argued that a quantitative model that would provide statistical data 
would be a better fit for AiP’s initial accreditation process. The model borrowed from LaRoche 
would have to wait until accreditation was more firmly established. While neither I nor the other 
English faculty enthusiastically supported this decision, we conceded and embarked upon a 
program of quantitative, rubric-based assessment. 

The focus of the quantitative assessment was our English Composition I course. Rubrics 
were generated that attempted to represent the holistic writing experience in six categories: 
Writing Process, Structure, Use of Sources, Use of Standard Rules of English Grammar, 
Understanding Reading and Writing as Interconnected Processes, and Use of Critical Thinking 
through Argument or Narrative. Deciding on these rubric categories and the attendant wording 
of each was a time-consuming and sometimes nettlesome process. The ideas and subsequent 
revisions of all faculty members, full and part-time, were considered. After months of debate 
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and multiple drafts, the rubric was approved by the faculty and particular rubric points such as 
“Writing as a Process” and “Reading and Writing as Interconnected Processes” were singled out 
for special examination, using the Nichols’ trademark “five column” method. Data were 
gathered and analyses written and discussed. However, as my colleagues mention here, the 
faculty found the process of quantitative assessment decontextualized and impersonal, despite all 
the work we had done to generate the rubrics. At about this time, the college was also in the 
process of opening a writing center and I had suggested my former colleague from La Roche, 
April Sikorski, to be its first director. Almost immediately we began talking about how to 
recreate the more qualitative program assessment we had experienced at LRC. April was now an 
Indiana University of Pennyslvania Ph.D. student in composition working on a dissertation on 
writing assessment and brought what she was learning to her work at AiP April challenged the 
rubric assessment, hoping to implement a more thoughtful and interactive qualitative assessment 
procedure, which would have faculty discussing how and why they grade portfolios in order to 
deepen mutual understanding of grades between faculty. Unfortunately, it was argued that the 
initial quantitative approach should be allowed to continue long enough to allow statistically 
significant data to be collected, and once again qualitative assessment was deferred for a more 
opportune moment. 

When Krystia Nora joined the AiP faculty in July 2007 to be the lead English faculty and 
Writers’ Center Coordinator, she came with experience in qualitative research and agreed with 
April that qualitative assessment could best examine the complex processes and interactions of 
teaching writing at AiP. With Krystia’s voice, we achieved a critical mass of three different 
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writing professionals suggesting the possible usefulness of qualitative assessment, and our 
associate dean, Maria Boada, finally allowed and even encouraged us to make our experiment. 

Krystia discusses how we came to use our specific qualitative methodology, but here is 
the list that ultimately convinced the administration to allow us to try a new approach: 

* Qualitative assessment could allow the faculty to talk about their grading procedures and 
what we valued in writing in greater depth, while rubric assessment minimized discussion 
once categories were chosen. 

* Qualitative assessment could let the faculty learn from each other and teach each other, 
while rubric assessment had faculty judge each other's students and courses. This, as it 
turned out, was hurting morale, as well as not providing a deeper understanding of each 
other's practices. 

* Qualitative assessment could facilitate deeper understanding of the intricacies of teaching 
of writing, while rubric assessment simplified success or failure in teaching of writing 
into six rubric points; even if these rubric points were well thought-out—we couldn’t 
hope to cover all that a teacher might expect in a writing assignment or course by 
examining six basic elements. 

* Qualitative assessment could better represent the subjective realities of each instructor’s 
course and his/her assignments, could provide a better record of the courses being taught, 
and (in a more complex way than rubric assessment) could better guide the participating 
instructors in revising their courses. 
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In short, while rubric assessment would show whether certain minimum standards were being 
met in the course, to the degree that the assessment was accomplished without robotic tiredness 
or subjective evaluations skewing results, we hoped that qualitative assessment could provide 
records that would be more accurate than rubric assessment in that we would not only show 
sample portfolios of classes, but include instructor analysis of the grading procedures for these 
classes—analysis done by engaged participants. Furthermore, qualitative assessment could 
provide an opportunity for, and record of, faculty development through transcripts and later 
analysis of instructor discussions about these courses, their portfolios, and the grading 
procedures. 
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